Challenges in Automated Model-Based HMI Testing

We describe our approach to automated model-based HMI testing. The paper is divided into two parts. In the first part, we summarize the current status of our work. In the second part, we describe a number of research areas that need to be worked on in order to achieve true model-based HMI test automation. 1 Test Automation in the HMI Domain The task of test automation involves two subtasks: (1) automated test case selection, and (2) automated test case execution. The long-term goal of test automation is to test against a complete formal specification of the unit under test. To be useful for test automation, a specification must comprise the static states and the dynamic behavior of the unit under test. The specification must be formal and sufficiently detailed in order to allow for automatic processing. We call such a comprehensive and formal specification a “model.” At present, such models are, in general, not available in the HMI domain. The design of the syntax and semantics of an appropriate specification language (i.e., modeling language) needs to take into account the requirements of automated testing. In order to be able to state these requirements, we need to start gaining experience with automated model-based testing, which in turn depends on the availability of models in the first place. In order to escape this chicken-or-egg situation, we have taken an intermediate step, in which the tests are performed not against the formal model of the HMI but against a prototype implementation of the HMI. This intermediate step is the topic of the next subsection. 1.1 Intermediate step toward test automation The key to prototype-based test automation is that the prototype implementation is used as a stand-in for the formal HMI model, which is not yet available. Instead of testing the HMI embedded control unit (the unit under test, henceforth called the HMI ECU) against the formal model, we directly compare the states and the behavior of the HMI ECU against the states and the behavior of the prototype implementation. In the following paragraphs, we briefly describe our rapid HMI prototyping framework FLUID (“Flexible User Interface Development”) and its role in the intermediate step toward model-based test automation. For a more-detailed description of FLUID, see [GES04].